1. Field of the Invention
The present invention relates to an image processing device for dividing an image into a plurality of image blocks, encoding the image blocks into image data and decoding the image data back into the image blocks. More particularly, the present invention relates to an image processing device which performs a discrete cosine transform (hereinafter, abbreviated as xe2x80x9cDCTxe2x80x9d) and an inverse discrete cosine transform (hereinafter, abbreviated as xe2x80x9cIDCTxe2x80x9d).
2. Description of the Related Art
As a method for compressing and decompressing a moving image, those employing DCT and IDCT have been commonly known. In such a method, a two-dimensional moving image is divided into square-shaped blocks each including Nxc3x97N pixels (hereinafter, referred to simply as xe2x80x9cimage blocksxe2x80x9d). The image data is compressed or decompressed by performing DCT or IDCT for each block of image data. By decomposing the image data using DCT, low frequency components, essential for reproduction of the image, can be extracted from the image data, since an actual image (or picture) contains only a small amount of high frequency components (e.g., those of the outline of an object). Based on this, image blocks can be compressed into image data.
The DCT and IDCT operations can be represented by the following Expressions (1) and (2), respectively:                               F          ⁡                      (                          u              ,              v                        )                          =                              2            N                    ⁢                                    ∑                              x                =                0                                            N                -                1                                      ⁢                                          ∑                                  y                  =                  0                                                  N                  -                  1                                            ⁢                                                C                  ⁡                                      (                    x                    )                                                  ⁢                                  C                  ⁡                                      (                    y                    )                                                  ⁢                                  F                  ⁡                                      (                                          x                      ,                      y                                        )                                                  ⁢                cos                ⁢                                                                            (                                                                        2                          ⁢                                                      xe2x80x83                                                    ⁢                          u                                                +                        1                                            )                                        ⁢                    x                    ⁢                                          xe2x80x83                                        ⁢                    π                                                        2                    ⁢                                          xe2x80x83                                        ⁢                    N                                                  ⁢                cos                ⁢                                                                            (                                                                        2                          ⁢                                                      xe2x80x83                                                    ⁢                          v                                                +                        1                                            )                                        ⁢                    y                    ⁢                                          xe2x80x83                                        ⁢                    π                                                        2                    ⁢                                          xe2x80x83                                        ⁢                    N                                                                                                          (        1        )                                          F          ⁡                      (                          x              ,              y                        )                          =                              2            N                    ⁢                                    ∑                              u                =                0                                            N                -                1                                      ⁢                                          ∑                                  v                  =                  0                                                  N                  -                  1                                            ⁢                                                C                  ⁡                                      (                    u                    )                                                  ⁢                                  C                  ⁡                                      (                    v                    )                                                  ⁢                                  F                  ⁡                                      (                                          u                      ,                      v                                        )                                                  ⁢                cos                ⁢                                                                            (                                                                        2                          ⁢                                                      xe2x80x83                                                    ⁢                          x                                                +                        1                                            )                                        ⁢                    u                    ⁢                                          xe2x80x83                                        ⁢                    π                                                        2                    ⁢                                          xe2x80x83                                        ⁢                    N                                                  ⁢                cos                ⁢                                                                            (                                                                        2                          ⁢                                                      xe2x80x83                                                    ⁢                          y                                                +                        1                                            )                                        ⁢                    v                    ⁢                                          xe2x80x83                                        ⁢                    π                                                        2                    ⁢                                          xe2x80x83                                        ⁢                    N                                                                                                          (        2        )            
where N denotes the number of pixels in a row or a column in one image block, whereby the total number of pixels in the block is Nxc3x97N;
F(u,v) denotes image data obtained by DCT, wherein u and v represent a location of the data within the block; and
f(x,y) denotes image data obtained by IDCT, wherein x and y represent a location of the data within the block.
C(k) in Expressions (1) and (2) can be represented by Expression (3) below.                               C          ⁡                      (            k            )                          =                  {                                                                                          1                                          2                                                        ⁢                                      xe2x80x83                                    ⁢                                      (                                          k                      =                      0                                        )                                                                                                                        1                  ⁢                                      xe2x80x83                                    ⁢                                      (                                          k                      =                                              1                        ∼                        7                                                              )                                                                                ⁢                      xe2x80x83                                              (        3        )            
As is apparent from comparison between Expressions (1) and (2), DCT and IDCT are substantially the same transform operations, and thus can be implemented with the same circuit configuration by changing coefficients. Therefore, while IDCT will be mainly discussed in the following description, such discussion applies also to DCT.
Two-dimensional IDCT, as represented by Expression (2), is typically implemented by twice performing one-dimensional IDCT (as represented by Expression (5) below). Expression (5) is derived from Expression (2) in such a manner, as in the following Expression (4).                               f          ⁡                      (                          x              ,              y                        )                          =                                            2              N                                ⁢                                    ∑                              v                =                0                                            N                -                1                                      ⁢                                          C                ⁡                                  (                  v                  )                                            ⁢              cos              ⁢                                                                    (                                                                  2                        ⁢                        y                                            +                      1                                        )                                    ⁢                  v                  ⁢                                      xe2x80x83                                    ⁢                  π                                                  2                  ⁢                                      xe2x80x83                                    ⁢                  N                                            ⁢                              (                                                                            2                      N                                                        ⁢                                                            ∑                                              u                        =                        0                                                                    N                        -                        1                                                              ⁢                                                                  C                        ⁡                                                  (                          u                          )                                                                    ⁢                                              F                        ⁡                                                  (                                                      u                            ,                            v                                                    )                                                                    ⁢                      cos                      ⁢                                                                                                    (                                                                                          2                                ⁢                                x                                                            +                              1                                                        )                                                    ⁢                          u                          ⁢                                                      xe2x80x83                                                    ⁢                          π                                                                          2                          ⁢                                                      xe2x80x83                                                    ⁢                          N                                                                                                                    )                                                                        (        4        )                                          f          ⁡                      (            k            )                          =                                            2              N                                ⁢                                    ∑                              n                =                0                                            N                -                1                                      ⁢                                          C                ⁡                                  (                  n                  )                                            ⁢                              F                ⁡                                  (                                      n                    ,                    k                                    )                                            ⁢              cos              ⁢                                                                    (                                                                  2                        ⁢                        k                                            +                      1                                        )                                    ⁢                  n                  ⁢                                      xe2x80x83                                    ⁢                  π                                                  2                  ⁢                  N                                                                                        (        5        )            
The one-dimensional IDCT of Expression (5) is repeated twice as follows. First, the one-dimensional IDCT is performed along the row (horizontal) direction, and then the one-dimensional IDCT along the column (vertical) direction is performed for the transform results, thereby obtaining a result which is equivalent to what is obtained by a single two-dimensional IDCT operation.
The one-dimensional IDCT, or Expression (5) above, is a simple product sum operation using a cosine function as a coefficient. Therefore, the circuit configuration required for implementing Expression (5) is relatively simple, and two-dimensional IDCT can thus be implemented more easily. Such a technique of repeating a one-dimensional transform twice instead of performing a single two-dimensional transform operation is disclosed in Japanese Laid-open Publication Nos. 7-200539 and 8-44709.
FIG. 19 schematically illustrates an image processing device employing IDCT based on the standard image compression/decompression method, MPEG. The image processing device receives encoded image data by image blocks each including Nxc3x97N pixels. The image data is further grouped in macroblocks each including up to six data blocks (respectively for luminance data, chromaticity data, and the like). Thus, a macroblock including a plurality of data blocks is input for one image block including Nxc3x97N pixels. Each data block is passed on from a VLD (Variable Length Decoding) section 101 to an IS (Inverse Scan) section 102, an IQ (Inverse Quantization) section 103, an IDCT section 104 and then to an MC (Motion Compensation) section 105. A certain operation is performed for the transferred data block at each section.
Each of the VLD section 101, the IS section 102, the IQ section 103 and the IDCT section 104 processes one data block at a time, and does so only after the preceding section (i.e., a section which processes the block immediately before the subject section) completely processes that particular block. The last section, i.e., the MC section 105, first receives all data blocks for the macroblock, and then performs an MC operation between the newly-received macroblock of data and the preceding macroblock of data which is input from a memory section 106, thereby creating and outputting image data corresponding to the image block of Nxc3x97N pixels.
A control section 107 generally controls the sections 101 to 105. Since the sections 101 to 104 each require a different amount of time for processing one data block, while the last section, i.e., the MC section 105, processes data by macroblocks, the control section 107 successively provides respective operation timings for the sections 101 to 105.
FIG. 20 illustrates a configuration of the IDCT section 104. The IDCT section 104 includes two one-dimensional IDCT sections 111 and 112, an inversion memory 113 provided therebetween and a control section 114.
The IDCT section 104 operates as follows. The one-dimensional IDCT section 111 performs one-dimensional IDCT for a data block. The transform result is temporarily stored in the inversion memory 113. Then, the one-dimensional IDCT section 112 performs one-dimensional IDCT for the stored transform result, thereby outputting a result which is equivalent to what is obtained by a single two-dimensional IDCT operation. The control section 114 generally controls the sections 111 to 113.
FIG. 21 is a timing diagram illustrating the operation timings of the respective sections 101, 103, 104 and 105 illustrated in FIG. 19. The IS section 102 is omitted in FIG. 21 since the operation thereof is negligibly short in time compared to those of the other sections.
As is apparent from this timing diagram, the VLD section 101 first processes a first data block B1. After the VLD section 101 completely processes the block B1, the IQ section 103 starts to process the block B1. Similarly, after the IQ section 103 completely processes the block B1, the IDCT section 104 starts to process the block B1. Then, after the IDCT section 104 completely processes the block B1, the sections 101 to 104 successively process a second data block B2, after which the sections 101 to 104 process a third data block B3 in the same manner. After the first to third data blocks B1 to B3, which correspond to one macroblock in this instance, have all been processed by the respective sections 101 to 104, the MC section 105 starts processing the macroblock of data.
Since one data block is processed successively by the VLD section 101, the IQ section 103 and the IDCT section 104, the control section 107 has to control the operation timings of the respective sections 101 to 104. Moreover, since the MC section 105 adds together the data block from the memory section 106 and the data block from the IDCT section 104, the control section 107 has to match the respective timings of the data blocks to be input to the MC section 105.
However, when the control section 107 is responsible for all such timing controls, the control section 107 becomes complicated.
Also in the IDCT section 104 illustrated in FIG. 20, since the control section 114 is responsible for controlling the operation timings of the one-dimensional IDCT sections 111 and 112, the control section 114 is complicated.
It is possible to provide only a single one-dimensional IDCT section and to process the same data block twice with that one-dimensional IDCT section. In such a case, however, the control section 114 becomes even more complicated.
Moreover, as can be seen from the timing diagram of FIG. 21, there are time gaps t1, t2, . . . , resulting between two operations performed by the IDCT section 104, which requires the longest processing time. It is apparent that such time gaps are wasteful and thus lengthen the overall processing time.
According to one aspect of this invention, an image processing device includes a plurality of processing sections for successively receiving and decoding a plurality of data blocks, which have been obtained by encoding a plurality of image blocks of an image. The plurality of processing sections include an inverse discrete cosine transform processing section for performing two-dimensional inverse discrete cosine transform. When one of the processing sections is unable to receive the data block, the one of the processing sections sends a busy signal to preceding one of the processing sections. When one of processing sections receives the busy signal, the one of the processing sections discontinues data block transfer to following one of the processing sections.
In one embodiment of the invention, the inverse discrete cosine transform processing section includes: a memory section for temporarily storing a plurality of data blocks; an operation section for successively performing an inverse discrete cosine transform for the data blocks stored in the memory section; and a control section for successively storing/erasing the data blocks in/from the memory section, the control section sending a busy signal to preceding one of the processing sections when the memory section is filled with the data blocks.
In one embodiment of the invention, the control section erases one of the data blocks stored in the memory section which has been processed so that a new data block may be stored in the memory section, while another one of the data blocks in the memory section is being processed by the operation section.
In one embodiment of the invention, the memory section includes first and second memory sections each for temporarily storing a plurality of data blocks. The operation section includes first and second operation sections each for performing a one-dimensional inverse discrete cosine transform. Each one of the data blocks is first stored in the first memory section, the first operation section performing a one-dimensional inverse discrete cosine transform for the data block, after which the data block is transferred to the second memory section, the second operation section performing a one-dimensional inverse discrete cosine transform for the data block, thus performing a two-dimensional inverse discrete cosine transform for the data block.
In one embodiment of the invention, the control section includes first and second control sections respectively for controlling the first and second operation sections. The first control section erases one of the data blocks stored in the first memory section which has been processed so that a new data block may be stored in the first memory section, while another one of the data blocks in the first memory section is being processed by the first operation section. The second control section erases one of the data blocks stored in the second memory section which has been processed so that a new data block may be stored in the second memory section, while another one of the data blocks in the second memory section is being processed by the second operation section.
In one embodiment of the invention, the control section includes first and second control sections respectively for controlling the first and second operation sections. The first control section sends a busy signal to preceding one of the processing sections while the first and second memory sections are both filled with data blocks.
In one embodiment of the invention, the second control section sends a busy signal to the first control section while the second memory section is filled with data blocks.
In one embodiment of the invention, each of the processing sections is provided with a memory section for storing at least one data block. One of the processing sections sends a busy signal to preceding one of the processing sections while the memory section of the one of the processing sections is filled with data blocks.
In one embodiment of the invention, the processing sections includes: a variable length decoding section; an inverse scan section; an inverse quantization section; an inverse DCT section; and a motion compensation section.
In one embodiment of the invention, each of the variable length decoding section, inverse scan section, the inverse quantization section and the inverse DCT section is provided with a memory section for storing at least one data block. One or more of the variable length decoding section, inverse scan section, the inverse quantization section and the inverse DCT section sends a busy signal to preceding one of the processing sections while the memory section of the one of the sections is filled with data blocks.
In one embodiment of the invention, the motion compensation section includes a memory section for storing a plurality of data blocks, and sends a busy signal to preceding one of the processing sections while the memory section of the motion compensation section is filled with a plurality of data blocks.
According to another aspect of this invention, an image processing device includes a plurality of processing sections for successively receiving and decoding a plurality of data blocks, which have been obtained by encoding a plurality of image blocks of an image, the plurality of processing sections including an inverse discrete cosine transform processing section for performing a two-dimensional inverse discrete cosine transform. The inverse discrete cosine transform processing section includes: a memory section for temporarily storing a plurality of data blocks; an operation section for successively performing an inverse discrete cosine transform for the data blocks stored in the memory section; and a control section for erasing one of the data blocks stored in the memory section which has been processed so that a new data block may be stored in the memory section, while another one of the data blocks in the memory section is being processed by the operation section.
In one embodiment of the invention, the memory section includes first and second memory sections each for temporarily storing a plurality of data blocks; the operation section includes first and second operation sections each for performing a one-dimensional inverse discrete cosine transform; and each one of the data blocks is first stored in the first memory section, the first operation section performing a one-dimensional inverse discrete cosine transform for the data block, after which the data block is transferred to the second memory section, the second operation section performing a one-dimensional inverse discrete cosine transform for the data block, thus performing a two-dimensional inverse discrete cosine transform for the data block.
In one embodiment of the invention, the control section includes first and second control sections respectively for controlling the first and second operation sections. The first control section erases one of the data blocks stored in the first memory section which has been processed so that a new data block may be stored in the first memory section, while another one of the data blocks in the first memory section is being processed by the first operation section. The second control section erases one of the data blocks stored in the second memory section which has been processed so that a new data block may be stored in the second memory section, while another one of the data blocks in the second memory section is being processed by the second operation section.
Thus, the invention described herein makes possible the advantage of providing an image processing device which can reduce the overall processing time with simple control.